Estimation

Various models were tried for this problem, with the exception of deep neural networks, since tensorflow and pytorch were forbidden for the project/competition.

The final model found to be most optimal was a StackingClassifier with base estimators AdaBoostClassifier and ExtraTreesClassifier, and final estimator XGBoostClassifier. GridSearchCV was used for hyperparameter tuning.

xgb = XGBClassifier(max_depth=8, learning_rate=0.01, colsample_bytree=0.3, objective="binary:logistic", random_state=42)
ada = AdaBoostClassifier(n_estimators=49, random_state=0)
et = ExtraTreesClassifier(n_estimators=1000, random_state=0)
estimators = [('ada', ada), ('et', et)]
classifier = StackingClassifier(estimators=estimators, final_estimator=xgb, passthrough=True)

classifier.fit(x_train, y_train)
y_pred = classifier.predict(x_test)